Learning with Clustering Structure

We study supervised learning problems using clustering constraints to impose structure on either features or samples, seeking to help both prediction and interpretation. The problem of clustering features arises naturally in text classification for instance, to reduce dimensionality by grouping words together and identify synonyms. The sample clustering problem on the other hand, applies to multiclass problems where we are allowed to make multiple predictions and the performance of the best answer is recorded. We derive a unified optimization formulation highlighting the common structure of these problems and produce algorithms whose core iteration complexity amounts to a k-means clustering step, which can be approximated efficiently. We extend these results to combine sparsity and clustering constraints, and develop a new projection algorithm on the set of clustered sparse vectors. We prove convergence of our algorithms on random instances, based on a union of subspaces interpretation of the clustering structure. Finally, we test the robustness of our methods on artificial data sets as well as real data extracted from movie reviews.Comment: Completely rewritten. New convergence proofs in the clustered and sparse clustered case. New projection algorithm on sparse clustered vector

Convex Relaxations for Permutation Problems

Seriation seeks to reconstruct a linear order between variables using unsorted, pairwise similarity information. It has direct applications in archeology and shotgun gene sequencing for example. We write seriation as an optimization problem by proving the equivalence between the seriation and combinatorial 2-SUM problems on similarity matrices (2-SUM is a quadratic minimization problem over permutations). The seriation problem can be solved exactly by a spectral algorithm in the noiseless case and we derive several convex relaxations for 2-SUM to improve the robustness of seriation solutions in noisy settings. These convex relaxations also allow us to impose structural constraints on the solution, hence solve semi-supervised seriation problems. We derive new approximation bounds for some of these relaxations and present numerical experiments on archeological data, Markov chains and DNA assembly from shotgun gene sequencing data.Comment: Final journal version, a few typos and references fixe

Iterative hard clustering of features

We seek to group features in supervised learning problems by constraining the prediction vector coefficients to take only a small number of values. This problem includes non-convex constraints and is solved using projected gradient descent. We prove exact recovery results using restricted eigenvalue conditions. We then extend these results to combine sparsity and grouping constraints, and develop an efficient projection algorithm on the set of grouped and sparse vectors. Numerical experiments illustrate the performance of our algorithms on both synthetic and real data sets

Relaxations convexes et spectrales pour les problèmes de reconstruction de phase, seriation et classement

Optimization is often the computational bottleneck in disciplines such as statistics, biology, physics, finance or economics. Many optimization problems can be directly cast in the well- studied convex optimization framework. For non-convex problems, it is often possible to derive convex or spectral relaxations, i.e., derive approximations schemes using spectral or convex optimization tools. Convex and spectral relaxations usually provide guarantees on the quality of the retrieved solutions, which often transcribes in better performance and robustness in practical applications, compared to naive greedy schemes. In this thesis, we focus on the problems of phase retrieval, seriation and ranking from pairwise comparisons. For each of these combinatorial problems we formulate convex and spectral relaxations that are robust, flexible and scalable.L’optimisation s’avère souvent essentielle dans de nombreuses disciplines: statistiques, biologie, physique, finance ou encore économie. De nombreux problèmes d’optimisation peuvent être directement formulés dans le cadre de l’optimisation convexe, un domaine très bien étudié. Pour les problèmes non convexes, il est souvent possible d’écrire des relaxations convexes ou spectrales, i.e., d’établir des schémas d’approximations utilisant des techniques convexes ou spectrales. Les relaxations convexes et spectrales fournissent en général des garanties sur la qualité des solutions associées. Cela se traduit souvent par de meilleures performances et une plus grande robustesse dans les applications, par rapport à des méthodes gloutonnes naïves. Dans ce manuscript de thèse, nous nous intéressons aux problèmes de reconstruction de phase, de sériation, et de classement à partir de comparaisons par paires. Nous formulons pour chacun de ces problèmes des relaxations convexes ou spectrales à la fois robustes, flexibles, et adaptées à de grands jeux de données

SerialRank: spectral ranking using seriation

We describe a seriation algorithm for ranking a set of n items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder this matrix and construct a ranking. We first show that this spectral seriation algorithm recovers the true ranking when all pairwise comparisons are observed and consistent with a total order. We then show that ranking reconstruction is still exact even when some pairwise comparisons are corrupted or missing, and that seriation based spectral ranking is more robust to noise than other scoring methods. An additional benefit of the seriation formulation is that it allows us to solve semi-supervised ranking problems. Experiments on both synthetic and real datasets demonstrate that seriation based spectral ranking achieves competitive and in some cases superior performance compared to classical ranking methods

Phase retrieval for imaging problems

International audienceWe study convex relaxation algorithms for phase retrieval on imaging problems. We show that structural assumptions on the signal and the observations, such as sparsity, smoothness or positivity, can be exploited to both speed-up convergence and improve recovery performance. We detail experimental results in molecular imaging problems simulated from PDB data

Spectral Ranking using Seriation

International audienceWe describe a seriation algorithm for ranking a set of $n$ items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder this matrix and construct a ranking. We first show that this spectral seriation algorithm recovers the true ranking when all pairwise comparisons are observed and consistent with a total order. We then show that ranking reconstruction is still exact even when some pairwise comparisons are corrupted or missing, and that seriation based spectral ranking is more robust to noise than other scoring methods. An additional benefit of the seriation formulation is that it allows us to solve semi-supervised ranking problems. Experiments on both synthetic and real datasets demonstrate that seriation based spectral ranking achieves competitive and in some cases superior performance compared to classical ranking methods

Spectral Ranking using Seriation

International audienceWe describe a seriation algorithm for ranking a set of $n$ items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder this matrix and construct a ranking. We first show that this spectral seriation algorithm recovers the true ranking when all pairwise comparisons are observed and consistent with a total order. We then show that ranking reconstruction is still exact even when some pairwise comparisons are corrupted or missing, and that seriation based spectral ranking is more robust to noise than other scoring methods. An additional benefit of the seriation formulation is that it allows us to solve semi-supervised ranking problems. Experiments on both synthetic and real datasets demonstrate that seriation based spectral ranking achieves competitive and in some cases superior performance compared to classical ranking methods

Blefaroespasmo essencial benigno Benign essential blepharospasm

Blefaroespasmo essencial benigno é distonia focal caracterizada por espasmos involuntários progressivos do músculo orbicular oculi e músculos da região superior da face (corrugador e procerus). As contrações forçadas e crônicas dos músculos perioculares tornam o paciente debilitado e levam a alterações funcionais e cosméticas das pálpebras. O tratamento inclui uma variedade de modalidades e medicações orais que apresentam eficácia limitada. Injeções de toxina botulínica tipo A têm apresentado bons resultados temporários, enquanto a miectomia periocular tem demonstrado bons resultados em longo prazo.<br>Essential blepharospasm is a focal dystonia characterized by progressive involuntary spasms of the orbicularis oculi and upper facial (corrugator and procerus) muscles. Chronic, forceful contractions of the periocular muscles is debilitating for the patient and leads to functional and cosmetic eyelid deformities. Treatment has included a variety of modalities and oral medications that are of limited efficacy. Botulinum-A injections have yielded the best temporary relief from this disorder, while the periorbital myectomy operation has been shown to give the best long-term results